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ith the Charter Schools Act 
V / of 1992, California began 
allowing for the creation of a 
new type of public school. The school’s 
organizers would operate more or less 
independently under a charter — or 
performance agreement — negotiated with 
a school district or county office of educa- 
tion. Groups could start new schools from 
scratch or convert existing schools to char- 
ter status. A charter school would be 
subject to fewer restrictions but be more 
accountable for student achievement. The 
schools were to meet student performance 
goals as specified in their charter. 

Today about 500 charter schools 
operate throughout California. Some 
have been around for more than 10 years, 
while others received their charters only 
recently. While most operate independ- 
ently, a few are under the auspices of 
larger organizations, some of which are 
for profit. 

In recent years, a number of studies 
have attempted to assess the effectiveness 
of the charter experiment, particularly its 
impact on students’ academic achieve- 
ment. These studies come to very 
different conclusions depending on the 
schools, the timeframe, and the perform- 
ance measures they analyze. Charter 



advocates and opponents alike watch 
these studies closely, with both sides 
looking for findings to bolster their case. 
Advocates — who see charter schools as a 
catalyst for broad reform — want to show 
that school-level decision making and 
freedom from most Education Code 
requirements allow charters to excel. 
Detractors, on the other hand, seek 
support for their argument that site-level 
control, combined with freedom from 
the regulations and formal oversight of a 
public bureaucracy, do not necessarily 
lead to better instruction and student 
performance and may even hurt educa- 
tional quality. 

This report begins with a discussion of 
why performance comparisons between 
charter and noncharter schools can be 
complicated. It also summarizes the most 
recent data from California’s assessment 
and accountability systems, comparing 
types of charter schools and looking at 
them within the context of the larger 
system. An overview of the often contra- 
dictory findings from prominent research 
studies illustrate how different experts 
address the comparison of academic 
performance among charter school 
students versus their peers in the regular 
system. This report also adds to that ongo- 



ing discussion with a new EdSource analy- 
sis that approaches the issue somewhat 
differently. Taken together, the informa- 
tion in this report raises questions 
Californians should be asking about the 
state’s charter schools and their perform- 
ance, and it illuminates some reasons why 
clear answers are so elusive. 



Exploring performance among charter 
schools is a challenge 



In California, the academic progress of 
students in individual charter schools can be 
examined using the same test-based meas- 
ures used for noncharters, however 
imperfect those may be. According to state 
statute, charter schools are required to 
participate in the state’s standardized testing 
programs. They are also evaluated using the 
same accountability systems the state and 
federal governments require of other public 
schools. (See the box on page 2.) Finding 



Key Findings 



On page 19 you will find a summary of the 
major conclusions included in this report 
regarding charter school performance in 
California. 
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State tests and accountability measures that apply to 
charter schools 



Charter schools are required to administer the following tests to their students and report the 

results to the state: 

• The Standardized Testing and Reporting (STAR) program, which includes the California Standards Tests 
(CSTs) in grades 2-11 plus the nationally referenced CAT/6 survey test (which replaced the SAT-9 in 
2003). Starting in 2005, only 3rd and 7th graders will take the CAT/6. 

• The California High School Exit Exam (CAHSEE). 

• The California English Language Development Test (CELDT) for English learners. 

In California, parents have a right to opt out of testing for their child. 

For purposes of school accountability, charter schools are ranked and/or evaluated based on the 

following: 

• The Academic Performance Index (API), which incorporates the results of STAR and CAHSEE to gener- 
ate Base and Growth API scores. 

• Adequate Yearly Progress (AYP), per federal requirements, based on the percent of students proficient 
in math and English on the CSTs, or the CAHSEE for high school students, plus additional indicators. 
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testing and accountability scores for indi- 
vidual schools is thus an easy matter. 

Generalizing about school perform- 
ance and comparing groups of schools 
presents a much more complex challenge, 
particularly if one is interested in 
comparing charters to other public 
schools. This is because of the constant 
turnover and increase in the number of 
schools and because the universe of char- 
ter schools is — in a variety of ways — quite 
different from noncharters. Within that 
universe, charter schools are also markedly 
different from each other. In addition, for 
various reasons they are substantially less 
likely to have full data sets. 

Constant change makes comparisons difficult 

While some schools open and close each 
year in California, the more than 9,000 
public schools are — as a group — rela- 
tively stable. The total number of schools 
changes little from year to year, and the 
mix of grade levels is on the whole fairly 



constant and predictable, as is the number 
of students these schools serve. 

Compare that to the charter school 
segment. Between 1993—94 and 
2004—05, the number of charter 
schools went from 31 to 518. During 
that same time, another 40 schools had 
their charters revoked and 95 others 
closed, according to California Depart- 
ment of Education (CDE) records. (See 
Figure I.) 

The constantly growing number of 
charter schools makes generalizations 
about charter school improvement over 
time difficult. A simple calculation — such 
as the percentage of high schools that met 
their state-determined growth targets on 
the Academic Performance Index (API) 
over five years — is only possible for a very 
small number of schools. (For an explana- 
tion of how the API works, see the box on 
page 4.) Presumably, as the number of 
long-lived charter schools increases, 
assessments of their improvement over 
time will become more meaningful. 

Charters often differ in grade configurations 

Charters are also more varied in their 
grade-level configurations. Most Cali- 
fornia charter schools, like most regular 
public schools, are configured as elemen- 
tary, middle, or high schools; but a 
substantial portion depart from that 
model. In 2003—04, a full 13% served 
all of grades kindergarten through 12, 
and another 15% could be categorized 
as nontraditional because they either 
offered only one grade, or they covered 
an unusual grade span that crossed over 
the traditional categories (e.g., K— 9 or 
I— II). Unconventional grade configu- 
rations are more common among 
charters, perhaps in part because some 
start-up charters add a grade each year as 
they become more established. 

For calculating API scores and rank- 
ing charter schools along with others, the 
state groups all these various configura- 
tions into three categories — elementary, 
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figure 1 



The number of charter schools* 
has increased dramatically in California 




Year 

10,761 23,228 30,977 39,624 48,101 68,685 99,048 113,956 132,643 158,942 167,764 181,734 

Enrollment' 

*To be counted among the schools open in a given year, a school must operate at a minimum from November 1 through February. 
••None of the achievement data in this report correspond to the 2004-05 school year, 
t Enrollment data is not available for a few schools each year. 



Data: California Department of Education (CDE) 
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middle, and high schools. (See the box on 
page 5.) While this approach makes it 
easier to compare large groups of schools, 
it may also create some misleading 
comparisons when school configurations 
do not fit the typical pattern. 

Charter schools’ student demographics 
appear somewhat different 

Comparing charters as a group to all 
noncharters can also be misleading 
because of student characteristics. 
Charter schools as a whole enroll 
somewhat different students than 
noncharters. Charter schools are less 
likely to serve Hispanic students, 
English learners, or students from low- 
income families. Elementary schools 
differ the most in this regard, while high 
schools differ the least. (It is important 
to note that the percentage of low- 
income students in charter schools may 
be undercounted because it is often 
based on eligibility for free/ reduced- 
priced meal programs, which many 
charter schools do not run.) 



When it comes to total school enroll- 
ments, however, the difference in size 
between charter and noncharter schools is 
most dramatic at the high school level. 
Elementary schools are more similar. 

Some research indicates that small 
schools are more effective, at least with 
some students. That represents another 
complicating factor because it is unclear 
to what extent school size, as opposed to 
charter status, may account for some 
differences in academic performance. 

Later in this report, analyses of 
elementary (page 9), middle (page II), 
and high schools (page 13) provide detailed 
comparisons of student demographics, 
school size, and teacher characteristics. 

Some authorities cite a less tangible 
difference between students in nonchar- 
ter versus charter schools. Demographic 
data cannot reveal the qualitative differ- 
ences in student attitude, motivation, or 
frustration with traditional school struc- 
tures that might differentiate charter 
students. Those differences could make a 
material difference in their performance. 



Status and operations differentiate 
charters from each other 

As stated earlier, the community of char- 
ter schools is quite diverse. Charters can 
be categorized in ways that help describe 
how they differ and illuminate some 
important distinctions vis-a-vis tradi- 
tional public schools. This report 
distinguishes among charters according to 
how they were established, how long they 
have been operating as a charter school, 
and whether they offer their instruction 
primarily in classroom settings. 

The first distinction is between conver- 
sion and start-up charters. Conversion 
charters were once regular public schools. 
Start-ups, on the other hand, began as char- 
ter schools. Presumably a conversion school 
decided to break away from state and/ or 
district office policies on curriculum, 
finance, the academic calendar, or other 
major issues. This new charter status was 
overlaid onto an existing school culture and 
ongoing staff and community relation- 
ships. By contrast, a start-up often 
represents one persons or groups brain- 
child that exists independent of a district 
context. It presumably begins as a new 
enterprise in which roles, relationships, and 
processes are all created. (These generaliza- 
tions may not fit every school officially 
categorized as a conversion or start-up.) 

Start-ups, which make up about 70% 
of all charter schools, vary more dramat- 
ically from noncharter public schools 
than conversions do. For example, the 
median noncharter school has 33% white 
students, the median conversion school 
has 45%, and the median start-up 55%. 
The relationships are similar for other 
student and school characteristics, with 
the charter schools generally having 
smaller student bodies and lower percent- 
ages of English learners, Hispanic 
students, and experienced and fully 
credentialed teachers. 

Second, there are established and new 
charters. It is reasonable to expect that a 
charter’s performance in its beginning 
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How the Academic Performance Index (API) works 



Since 1999, California has evaluated school performance using its Acade- 
mic Performance Index (API) system. The API score is a one-number 
summary of a school’s scores on various tests. The school also receives an 
API for each “numerically significant” subgroup of pupils categorized by 
ethnicity and poverty. 

The API system is organized into two-year cycles. Some time between January 
and March, each school receives a “Base API" score, based on its students’ 
performance on tests given in the spring of the prior year. It also receives 
growth targets for improvement. Students take state tests again in the late 
spring, and those scores are used to calculate a “Growth API” that schools 
receive in the fall, completing the two-year cycle. 

Student scores on the state’s standardized tests are used to create the 
index. The combination of tests used in the index has evolved along with the 
state testing system. Beginning with the 2002 Base APIs, the California stan- 
dards tests-which assess students’ mastery of the state's academic content 
standards-have played a dominant role in the API. Student performance on 
these tests is reported as meeting one of five performance levels: far below 
basic, below basic, basic, proficient, and advanced. 

An API score is based on the distribution of scores among the five performance 
levels, with various subjects and tests receiving differing weights in that calcu- 
lation. API scores can range from 200 to 1000. if all students score in the top 
performance band on all subtests, the API score will be 1000. The state set 
800 as the target score for all schools. 

The elements that make up the API have changed almost every year, which is 
one of several reasons why some criticize it. However, in each API cycle, the 
Growth API is calculated using the same elements as the Base API, ensuring 
that the Base/Growth results are comparable. For high schools, the California 
High School Exit Exam, which assesses mastery of standards in English 
language arts and math, was incorporated beginning with the 2002 Base API. 

Base API scores are used to compare or rank schools and to set growth 
targets. Schools are ranked with schools of the same type-elementary, 
middle, and high.* First, they are ranked against all their counterparts in the 
state, and next they are ranked against the 100 most similar schools, based 
largely on student demographics. For both types of rankings, they are clustered 
into 10 groups of roughly equal size known as “deciles.” The bottom 10% of 
each school type belongs to Decile 1, the second lowest 10% to Decile 2, and 
so on. A shorthand has developed around these rankings: a “7/5” school, for 



example, would be a school that received a state decile ranking of 7 and a 
“similar schools" ranking of 5. 

To achieve growth in its API score, a school (or subgroup) needs to decrease 
the percentage of pupils who score in the lower performance bands and 
increase the percentage who score in the higher bands. The API formula 
rewards growth from the bottom of the performance distribution more heavily. 
This creates an incentive for a school to work with its lowest-performing 
students. 

For a school with a Base API score below 800, the annual growth target is 5% 
of the difference between 800 and its Base score. For example, a school with 
a Base score of 500 would have a growth target of 15, which is 5% of the differ- 
ence between 500 and 800. Schools with Base scores of 800 and above are 
expected to maintain scores at that level. Subgroup growth targets are gener- 
ally 80% of the school’s target. 

The Growth API is one way to measure improvement from year to year. The 

API does not reflect individual students' test-score growth. It summarizes a 
school’s performance in one year (Base) and compares it to the school’s 
performance in the following year (Growth), but the groups of students at each 
grade level are different. For example, in a K-5 elementary school, the Base 
API includes scores of fifth graders who would no longer attend the school 
during the Growth API year. This diverges from the “gold standard” for assess- 
ing schools’ academic performance, which would be to follow the test-score 
growth of individual students overtime. 

Before 2002, some schools were not assigned APIs. Originally, California did 
not assign API scores to every school every year, and many charter schools 
were among those excluded. When the system was first created, a school did 
not receive an API if it had fewer than 100 test-takers or if its student body was 
predominantly at-risk students (e.g., continuation high schools, court schools, 
or other alternative schools). These schools instead qualified for the “Alterna- 
tive Schools Accountability Model." Further, schools with irregularities in the 
test administration, or that experienced a significant demographic change 
from the previous year, did not get an API score in the second year. 

(For a further explanation of the various components used to calculate the 
API, see www.edsource.org/edu_acc_api.cfm.) 

"“Small” schools-those with fewer than 100 test-takers— technically are a part of the Alternative 
Accountability system. Their API scores come with an asterisk to indicate that the scores may not be 
as accurate an indicator of the schools' “true" performance as it is for larger schools. Small schools 
do not affect the rankings, but they are given the ranking associated with their API scores. 



stage is not necessarily indicative of its 
full potential. For this report, EdSource 
somewhat arbitrarily defines conversion 
charters as “established” if they have been 



open for a full school year. Start-ups are 
deemed “established” if they have been 
open for two full years. Conversions and 
start-ups are treated differently because it 



is assumed that conversions, though they 
experience changes in governance and 
finance when they convert, generally still 
enjoy some continuity in students, staff 
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How schools are classified for the Academic Performance Index (API) 



For API purposes, most California public schools are ranked among schools of the same type-elemen- 
tary, middle, and high. However, schools have a great variety of grade spans, which necessitates rules for 
classifying schools into those three categories. 

The California Department of Education (CDE) generally classifies schools based on the largest number 
of grades a school has in “core" grade spans of K-5, 7-8, and 9-12. (Sixth grade is left out of the core 
grade spans because some view it as an elementary grade while others see it as a middle grade.) A K-8 
school, for example, would be considered an elementary school because it has six grades in the core 
elementary grades versus only two grades in the core middle grades. For schools with an equal number 
of grades in two or three core spans, the CDE assigns schools to a type based on the largest enrollment 
in a core grade span. This means that a hypothetical 4-10 school with 130 students in grades 4-5, 190 
students in grades 7-8, and 120 students in grades 9-10 would be deemed a middle school for purposes 
of the API because the middle grade span has the most students. Schools are assigned the same type for 
both years in a two-year API Base/Growth cycle. From one cycle to the next, however, their type may 
change based on the students that they serve. 



and facilities. In 2003—04, about 65% of 
charter schools would have been consid- 
ered “established” under EdSource’s 
definition. Established and new schools 
were similar in terms of student ethnicity 
and parent education levels, but the newer 
schools tended to have less experienced 
and credentialed teaching staffs and to be 
smaller. For example, the median 
noncharter had 620 students, the median 
established charter had 297, and the 
median new charter had 169. 

Finally, there are classroom-based and 
nonclassroom-based charters. In 2001 the 
Legislature established a definition of a 
nonclassroom-based charter school as one 
that does not require its pupils to be onsite 
under the direct supervision of a teacher 
for at least 80% of the instructional 
time — four school days out of five, for 
example. Schools that provide a substan- 
tial portion of their instruction through 
distance learning or independent study 
generally fit that definition. Charters with 
a traditional classroom system, which 
make up almost 70% of all charter 
schools, are considered classroom-based. 

Charters deemed nonclassroom- 
based were very similar to their 
classroom-based counterparts in total 
enrollments but had relatively large 
percentages of white students and low 
proportions of English learners. They 
also had a high rate of student “mobility” 
(median of 55% versus 15% in regular 
public schools), indicated by a relatively 
high percentage of students in their first 
year at the school (lowest grade excluded). 
There are many plausible explanations for 
this. For example, it could indicate that a 
different type of student is attracted to 
these schools. Perhaps students having 
trouble in conventional school settings 
use nonclassroom-based schools as a 
transition between more traditional 
schools. Such differences in the type of 
students served, which are not readily 
measured and reported to the state, 
should be kept in mind when comparing 



the achievement of these schools to other 
charters and mainstream noncharters. 

Variations in funding and other policies 
complicate comparisons 

Charter schools generally receive more 
money per pupil than districts receive in 
general purpose funds (revenue limit 
funding) but less than the total most 
districts receive when special purpose 
funds are considered. But wide variations 
in funding per district leave even this 
simple statement open to debate. 

It is clear that many charters forego 
extra funding they could receive because 
they want to avoid the administrative 
requirements of some state and federal 
programs. In a July 2003 evaluation of 
California’s charter schools, RAND 
found that charters were much less 
likely than comparable noncharters to 
participate in eight relatively large 
programs, including the federal Title I 
program for low-income students, K— 3 
Class Size Reduction, and Supplemen- 
tal Instruction. 

Many charter schools, especially 
start-ups, also spend at least a portion of 



their operating funds to cover the cost of 
facilities, a necessity few regular public 
schools face. Although charter schools 
have in recent years been given more 
avenues for securing facilities — or fund- 
ing for them — some are unable to take 
advantage of those options. For example, 
some of these funding avenues require 
that the school or at least its operators 
have a successful track record or that they 
serve primarily low-income students. 

On the other side of the equation, 
some charter schools enjoy substantial 
amounts of support from private sources. 
These funds usually come with fewer of 
the requirements attached to government 
funds. RAND reported that charter 
schools on average received $433 per 
student from these sources while compa- 
rable regular schools received $83. 

Within the charter community, 
schools differ in other important ways. 
Charters can choose to operate as finan- 
cially independent entities, getting their 
funds straight from the state. Or they 
can be financially dependent, with their 
chartering school district passing state 
funds through to them and helping 
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them with some management, adminis- 
trative, and/ or operational functions. 

Charter schools also vary in their 
employment agreements with teachers and 
other staff. At one end of the spectrum 
are those whose teachers are represented 
by the same union as their counterparts 
throughout their chartering district. At 
the other end are schools where teachers 
choose to forego union representation 
with its employment protections and 
collective bargaining rights. 

(For more details on the differences in 
funding, see EdSource’s June 2004 
report, Charter Schools in California: An 
Experiment Coming of Age f 

A substantial portion of charter schools do 
not have API data 

A final factor that makes assessments of 
charter school achievement complex is 
the substantial portion of schools with- 
out API data. Consider the charter 
school universe in 2003—04. That year, 
454 charter schools were open, but 82 
did not have 2004 Base API scores. Of 



those 82 schools: 12 were not in the 
California Department of Education’s 
API file; 23 were in the alternative 
schools accountability model, which 
does not use API scores to monitor 
performance; and 47 were listed in the 
API file but did not have scores for vari- 
ous reasons (explained below). Those 47 
schools constituted 11% of the charter 
schools listed. 

In contrast, of the states 8,766 
noncharter public schools, 7,813 were in 
the API system and 3% lacked scores. 
Another 953 schools were in the alterna- 
tive schools accountability model (ASAM). 

Among charter schools, 353 were 
listed in the 2004 Growth API file, 
which reports growth between the 2003 
Base and 2004 Growth APIs. Of those, 
80 schools (or 23%) lacked scores. In 
contrast, 6% of noncharters in the 2004 
Growth API file did not have data. Figure 
2 illustrates the wide variations in API 
data available based on school types. 
Note that almost half of nonclassroom- 
based schools are missing data. 



The California Department of Educa- 
tion lists a variety of reasons why schools 
do not receive a score, but there are two 
main reasons that charter schools are miss- 
ing data. One is that the schools did not test 
a sufficiently large group of students to 
have statistical meaning and preserve indi- 
vidual students’ privacy. The other is that 
the students tested were not sufficiently 
representative of the school as a whole, 
often due to a large percentage of parents’ 
excusing their children from STAR tests, 
upon which API scores are largely based. 

In addition, new, start-up charter 
schools do not receive their first API 
score until much of their second year of 
operation has passed because they do not 
participate in the state testing system 
until the end of their first year, and it 
takes about nine months for Base scores 
to come out. Effectively then, it is not 
until a charter’s third year that its first 
Growth API score, and thus its success 
under the state’s accountability system, 
has been reported. Of course, this is true 
for new noncharter schools as well. 



A comparison based on types of charters 



When test data is used as the measure of 
school performance, it can be done several 
ways. The most accurate approach is to 
compare the performance of a particular 
group of students over time. In California 
such a comparison cannot be done at the 
state level because individual student data 
is not available. California’s API system 
makes it possible, however, to do three 
other types of comparisons. 

Absolute comparisons ignore differ- 
ences in student characteristics as California 
does with its API rankings. These are 
released each year along with schools’ Base 
API scores. The state system only makes 
absolute comparisons among all schools 
that serve similarly-aged students, i.e., 
elementary, middle, and high school. 



Similar school comparisons look at 
schools whose students share similar char- 
acteristics. This approach acknowledges 
that school performance is highly corre- 
lated to students’ backgrounds, most 
notably the education level of their parents. 
Schools in California receive a “similar 
schools” ranking that serves this purpose. 

Comparisons over time examine a 
single school’s improvement in perform- 
ance. In California this involves a 
comparison of Base and Growth APIs in a 
two-year cycle. The state assigns each 
school a growth target that is a benchmark 
for its success and reports on whether each 
school met its target. (This kind of meas- 
ure is less meaningful when a school is 
high performing from the start). 



The following sections use these 
three types of comparisons to look at 
charter school performance. They exam- 
ine charter school performance by type, 
by grade level, and finally based on the 
state’s newly established performance 
criteria for charter renewal. The result is 
a rich tapestry of information that not 
only illuminates charter school perform- 
ance, but also makes clear how many 
questions remain. 

In California’s public schools, the 
state-adopted academic content stan- 
dards represent official expectations for 
the skills and knowledge students will 
master in school. The California stan- 
dards tests (CSTs) are used to measure 
students’ mastery of the standards. 
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figure 2 1 Percent of schools meeting 2004 API Growth Targets 



School Type 

(Pie charts: Percent 

with/without API data) 


2003-04 

Enrollment 


Number of 
Schools in 
API System 


Number of 
Schools with 
API Data 


Percent of Schools with 
API Data Meeting 2004 
API Growth Targets 




Yes 


No 


All Noncharter Schools 


5,931,819 


7,671 


7,237 


48% 


52% 


94 % 6% 












All Charter Schools 


143,264 


353 


273 


60% 


40% 




Charter Schools By Type 

Conversion 48,508 79 75 61% 39% 





106,116 


229 


188 


59% 


41% 


27,148 


124 


85 


61% 


39% 



Classroom-based 




Nonclassroom-based 




94,288 


252 


218 


64% 


36% 


48,976 


101 


55 


44% 


56% 



0 Schools with API data | Schools without API data 



Data: California Department of Education (CDE) 
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Students’ scores on the CSTs and other 
tests are summarized into Academic 
Performance Index (API) scores for the 
school as a whole and subgroups of 
students. (For a more detailed explana- 
tion of the API and its limitations, see 
the box on page 4.) 

Not everyone supports the state’s 
academic content standards or thinks that 
the CSTs accurately measure students’ 
mastery of those standards. Some see the 
API as an unreliable and arbitrary meas- 
ure of schools’ and subgroups’ 
achievement. But meeting API growth 
targets represents the state’s goal for all 
schools, including charters. As such, it is 
as fair and sound a measure of charter 
students’ achievement as it is for nonchar- 
ters. However, all charter schools are not 
alike, and the ways they differ seem to 
have some relationship to their ability to 
succeed based on the API measure. 

Missing data clouds comparisons of 
improvement by charter type 

Figure 2 shows the percentage of charter 
schools in the various categories that 
improved over time, meeting both school- 
wide and subgroup API growth targets in 
the 2003 Base/2004 Growth cycle. It 
also shows the number of schools held 
accountable in the API system and the 
number with actual API scores. A 
comparison of the two numbers shows 
that a substantial portion of some types 
of charter schools did not have API data, 
making it more difficult to compare 
performance in a meaningful way. 

Based on these data, conversion 
schools narrowly outperformed start-up 
schools, though it must be noted that the 
number of start-ups missing data was 
close to the total number of conversion 
schools. Classroom-based schools did 
substantially better in meeting growth 
targets than nonclassroom- based schools 
that had growth data, but nearly half of 
the nonclassroom-based schools lacked 
growth scores. Somewhat surprisingly, 
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Percent of schools meeting 2004 API growth targets, 
further disaggregated 



Established 
vs. New 


Classroom-based vs. 
Nonclassroom-based 


Conversion 
vs. Start-up 


Number of 
Schools 


Percent with 
Growth Data 
Meeting 
Growth Target 




In API 
System 


With 

Growth 

Data 


Yes 


No 


Noncharter Schools 


7,671 


7,237 


48% 


52% 


Charter Schools 


353 


273 


60% 


40% 




Classroom-based 


Conversion 


61 


60 


62% 


38% 


Established 


Start-up 


100 


90 


64% 


36% 


Charters 


Nonclassroom-based 


Conversion 


13 


10 


40% 


60% 




Start-up 


55 


28 


43% 


57% 




Classroom-based 


Conversion 


5 


5 


100% 


0% 


New Charters 


Start-up 


86 


63 


62% 


38% 


Nonclassroom-based 


Conversion 


0 


0 


N/A 


N/A 




Start-up 


33 


17 


47% 


53% 



Data: California Department of Education (CDE) 



EdSource 5/05 



new schools beat out established 
charters by a small margin. (To be consid- 
ered “established,” a conversion must 
have been open by 2001—02, and a 
start-up must have been open by 
2000-01.) 

Classroom-based schools consistently 
perform better 

Comparing the performance of charter 
schools by category is likely to interest 
policymakers who want to foster the 
creation of successful charter schools and 
discourage the less successful types. 
However, policy decisions based just on 
data for these broad categories could 
overreach. For example, conversion 
schools are outperforming start-ups, but 
are the conversion schools relying mostly 
on classroom-based instruction while 
start-ups are mostly nonclassroom-based? 
Cross-tabulating the various categories 
makes it possible to see beneath the 



surface comparison to address this type 
of question. 

In Figure 3, schools are first catego- 
rized as established or new, then 
subdivided into classroom-based or not 
and — within that — as conversions or 
start-ups. The table shows the percentage 
of each subcategory that improved over 
time, meeting its growth targets in 2004. 
The number of schools in each group 
should also be considered when compar- 
ing performance. 

From this analysis, it appears that the 
most successful charters in meeting 2004 
API growth targets were newer, classroom- 
based, conversion schools. But while all 
schools in that group met their targets, the 
group included only five schools. 

Classroom-based charters in other 
subcategories also did well, however. The 
three runner-up subcategories all offered 
classroom-based instruction, and all did 
better than noncharters, 48% of which 



met their growth targets. The least 
successful charters were established, 
nonclassroom-based, conversion schools. 
Only 40% met their targets, but three of 
13 were missing data. 

Given that classroom-based charters 
tended to perform well — and that very 
similar percentages of established and 
new charter schools offered classroom- 
based instruction — the similarity in 
performance between established and 
new charters does not seem as surpris- 
ing as it otherwise might. Similarly, the 
stronger performance of conversion 
schools as compared to start-ups may 
be partially explained by the fact that 
conversions were more likely than start- 
ups to offer classroom-based 
instruction (87% of schools with API 
scores versus 77%). 

Another factor affecting perform- 
ance is the grade levels that a school 
serves. For example, elementary schools 
have typically been more successful on 
the API, both in terms of overall scores 
and in the percentage of schools meeting 
growth targets. Therefore, comparisons 
of different types of charters, and 
between charters and noncharters, must 
take the grades served into considera- 
tion. The following section examines 
how elementary, middle, and high 
school charters, as defined in the API 
system, have performed. This looks at 
improvement over time, absolute 
comparisons, and similar schools 
comparisons. Before turning to 
performance per se, it is helpful to 
understand how charter schools at each 
grade level compare to noncharters. 
Thus each of the grade-level sections 
that follows provides that data. The 
performance of various subcategories of 
charters within those grade-level types is 
also discussed. 
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Elementary charter school performance is improving 



Key Characteristics of California Elementary Schoolst in 2003-04 

Statewide average: shows the percentage of elementaiy students or teachers in each category statewide. 

Median: shows the percentage for a “typical” school-one at the 50th percentile for a given characteristic. Equal numbers of schools 
have higher and lower percentages. 

Range for middle half: shows the percentages for the schools at the 25th and 75th percentiles. It provides a sense of how schools 
within each group vary and how charters and noncharters differ as well. 





Charter 

Schools (n=231) 


Noncharter 
Schools (n=5,427) 


Student Characteristics 


Median 


Median 


(Statewide Average) 


Range for Middle Half 


Range for Middle Half 


English Learners (29%) 


11%** 


24%* 




3%-50% 


9%-46% 


Hispanic (43%) 


17% 


39% 




8%-52% 


16%-70% 


White (36%) 


54% 


30% 




8%-76% 


8%-60% 


African American (8%) 


4% 


3% 




1%-U% 


l%-9% 


Asian (8%) 


1% 


3% 




0%-4% 


l%-9% 


Free/Reduced Price Meals (52%) 


26%* 


53% * 




2%-63% 


21%-78% 


Parents Not High School Grads (21%) 


4%* 


15%* 




1%-18% 


4%-32% 


One+ Parent a College Grad (18%) 


26%* 


16%* 




14%-36% 


8%-28% 


Student Mobility (20%) 


24% * 


17%* 




14%-45% 


12%-22% 


School Characteristics 


Enrollment (average size 566) 


222 


555 




123-422 


402-729 


Teachers Not Fully Credentialed (6%) 


13% 


3% 




0%-29% 


0%-8% 


Teachers in First Two Years (11%) 


20% 


8% 




10%-38% 


3%-14% 



Data: California Department of Education (CDE)— CBEDS EdSource 5/05 

flNCLUDES ONLY SCHOOLS IN THE API SYSTEM NOT THOSE IN THE ALTERNATIVE SCHOOLS ACCOUNTABILITY MODEL. 

*DATA MISSING FOR UP TO 20 % OF SCHOOLS. 

**DATA MISSING FOR AT LEAST 20 % OF SCHOOLS. 



The data indicate that elementary char- 
ter school students are less likely to 
come from the backgrounds most 
correlated with low student achieve- 
ment than their noncharter 
counterparts. The “typical” (median) 
elementary charter has about half as 
many English learners and students 
eligible for free/ reduced-price meals as 
well as a third as many students whose 
parents do not have a high school 
diploma. These charter schools are also 
about 50% smaller but have more 
students new to the school. They also 
have a much greater percentage of 
teachers who are less than fully creden- 
tialed and less experienced. 

Charter elementary schools surpass 
noncharters meeting growth targets 
for first time in 2004 

As with charter schools generally, the 
number of charter elementary schools 
has grown substantially, making the task 
of looking at their progress over time 
somewhat complex. In addressing the 
question of whether they are meeting the 
states expectations for improvement, the 
following statistics are important to keep 
in mind: 

• 202 charter elementary schools were in 
the API system of accountability in 
the 2003 Base/ 2004 Growth cycle. 

• 175 schools — 87% — have the data 
necessary to determine whether they 
met their growth targets in the most 
recent API cycle, compared to just 49 
schools in the 1999/2000 cycle. 

• Of those 175 schools, 58 were conver- 
sions and 117 were start-ups; 145 
were classroom-based and 30 were 
nonclassroom-based. 

• These 175 schools had 70,198 
students. 

Figure 4 compares elementary charter 
schools to noncharters. The data show 



their ability to meet the state’s expecta- 
tions for improvement over time based on 
API growth targets for the school as a 
whole and for all significant student 
subgroups. (See the box on page 4 for 
further explanation). 



In the last two cycles — since the state 
standards have played a predominant role 
in the index — elementary charter 
performance has improved. With the 
2004 Growth API, charters exceeded 
noncharters on this measure for the first 
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figure 4 I Elementary schools that made API growth targets 



Two-year API Base/Growth Cycle 





1999/00 


2000/01 


2001/02 


2002/03 


2003/04 


Number of charters in API 
accountability system 


56 


73 


96 


130 


202 


Number of charters with growth data 


49 


69 


89 


120 


175 


Percent of charters with growth data 
making growth targets 


69% 


49% 


52% 


75% 


57% 


Percent of noncharters with growth 
data making growth targets 


77% 


64% 


60% 


83% 


46% 



• It is more meaningful to compare percentages within individual API cycles because of the growing number of charter elementary 
schools and the changing index used to measure school performance (the API). 

• The bottom two rows reflect the schools with API data in both years of a given two-year API cycle. 



Data: California Department of Education (CDE) EdSource 5/05 



time. On average, these schools gained 
eight points while noncharter elementary 
schools gained four. This greater growth 
allowed 57% of elementary charters to 
meet their targets, while 46% of 
noncharters hit their marks. 

Based on absolute rankings, the 
elementary charter top performers 
were classroom-based 

The spring 2004 test results were also 
used to create new Base API scores. Based 
on those scores, schools were ranked 
among others serving the same grade 
levels and, separately, among 100 schools 
facing comparable challenges. 

For absolute comparisons, schools are 
ranked in 10 bands, called deciles, with 
each decile representing 10% of schools. 
The figures below reflect the percentages 
of charter elementary schools that placed 
in the highest, middle, and lowest range of 
deciles. It includes the 206 charters listed 
in the 2004 Base API that had a decile 
rank. To put the percentages in perspective, 
keep in mind that about 30% of schools 
overall occupy the top three deciles, 40% 
fall in deciles 4—7, and the remaining 30% 
are in the lower three deciles. 



• High-performing, deciles 8—10: 

26% (53 schools) 

• Mid-performing, deciles 4—7: 

39% (80 schools) 

• Low-performing, deciles 1—3: 

35% (73 schools) 

The charter schools in the top three 
deciles were predominantly established 
schools with classroom-based programs. 
Of the 13 charter elementary schools in 
Decile 10, all but one fit that description. 
Similarly, the Decile 8 and Decile 9 char- 
ter schools — of which there were 
40 — included 32 schools that were estab- 
lished and classroom-based. 

Interestingly, the charter elementary 
schools in the low-performing deciles 
represent a much wider range of configu- 
rations. Many classroom-based schools 
are among the lower deciles, but not 
disproportionately. New start-ups are 
over-represented in the lower perform- 
ance bands, however. 

Based on similar schools rankings, many 
elementary charters are not doing well 

The state uses the “similar schools” 
rankings to indicate how well a school is 
doing compared to the set of 100 



schools most like it in terms of student 
demographics, teacher qualifications, 
and a few other factors. A school with a 
similar schools rank of “ I ” scored in the 
bottom 10% of its comparable schools, 
and a school with a “10” scored in the 
top 10%. When looking across all of 
California’s public schools, each similar 
schools rank contains roughly 10% of 
schools. Therefore, if a large group of 
charter schools is performing on par 
with noncharters, roughly 30% will 
score in deciles 8—10, 40% will have a 
rank of 4—7, and the remaining 30% will 
score in deciles 1—3. Because schools 
with fewer than 100 test scores do not 
receive such rankings, the number of 
charter schools with such ranks in the 
2004 Base API is 142 (versus 206 with 
statewide decile ranks). The elementary 
charters with similar schools ranks scored 
as follows: 

• High-performing compared to similar 
schools, ranks 8—10: 

23% (33 schools) 

• Mid-performing compared to similar 
schools, ranks 4—7: 

23% (32 schools) 

• Low-performing compared to similar 
schools, ranks 1—3: 

54% (77 schools) 

Here again, conversion schools did 
much better than start-ups, and class- 
room-based charters substantially 
outperformed schools that were not 
classroom-based. Those that did very 
well relative to their comparison 
schools, scoring an “8” or above, were 
all classroom-based. Nonclassroom- 
based charter schools, on the other 
hand, consistently performed worse 
than the schools to which they were 
compared. Of the 26 nonclassroom- 
based schools, 18 received a similar 
school ranking of I, and none scored 
above a 6. 
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Charter middle schools are small in number but perform well 



Key Characteristics of California Middle Schoolst in 2003-04 

Statewide average: shows the percentage of middle students or teachers in each category statewide. 

Median: shows the percentage for a “typical” school-one at the 50th percentile for a given characteristic. Equal numbers of schools 
have higher and lower percentages. 

Range for middle half: shows the percentages for the schools at the 25th and 75th percentiles. It provides a sense of how schools 
within each group vary and how charters and noncharters differ as well. 





Charter 
Schools (n=43) 


Noncharter 
Schools (n=l,208) 


Student Characteristics 


Median 


Median 


(Statewide Average) 


Range for Middle Half 


Range for Middle Half 


English Learners (20%) 


11%** 


16%* 




5%-ll% 


6%-30% 


Hispanic (41%) 


22% 


36% 




12%-62% 


17%-63% 


White (37%) 


43% 


37% 




4%-64% 


ll%-62% 


African American (8%) 


5% 


4% 




1%-17% 


1%-10% 


Asian (8%) 


3% 


3% 




0%-7% 


1%-10% 


Free/Reduced Price Meals (41%) 


26%* 


40% 




3%-68% 


14%-65% 


Parents Not High School Grads (21%) 


5%* 


16%* 




2%-22% 


6%-32% 


One+ Parent a College Grad (20%) 


29% * 


18%* 




14%-42% 


10%-29% 


Student Mobility (19%) 


15%* 


13% * 




10%-52% 


10%-19% 


School Characteristics 


Enrollment (average size 956) 


213 


914 




104-393 


660-914 


Teachers Not Fully Credentialed (11%) 


19% 


6% 




3%-54% 


2%-15% 


Teachers in First Two Years (13%) 


25% 


11% 




12%-54% 


6%-17% 



Data: California Department of Education (CDE)— CBEDS EdSource 5/05 

f I ncludes only schools in the API system not those in the Alternative Schools Accountability Model. 

"Data missing for up to 20% of schools. 

"'Data missing for at least 20% of schools. 



As with elementary schools, charter 
middle schools tended to serve students 
from more advantaged backgrounds but 
with more uncredentialed and inexperi- 
enced teachers. The median charter 
middle school had about one-third fewer 
English learners, one-third fewer low- 
income students, and students with much 
better-educated parents. The typical char- 
ter middle school had three times as many 
teachers who were not fully credentialed 
and twice as many teachers in their first 
two years in the profession. The charter 
schools were also dramatically smaller, 
and the mobility of their student popula- 
tions was comparable to noncharters. 

The small group of charter middle schools 
showed impressive improvement from 
2003 to 2004 

Using the API to track over time the 
progress of charter middle schools as a 
group is a particular challenge because of 
the very small number of schools today 
and the even smaller number in existence 
in 1999. In the 2003 Base/ 2004 Growth 
API cycle: 

• 31 charter middle schools were in the 
API system of accountability. 

• 27 schools — or 87% — had the data 
necessary to determine whether they 
met their growth targets, compared to 
just 1 1 schools in the 1999/ 2000 cycle. 

• Of those 27 schools, nine were conver- 
sions and 18 were start-ups. All were 
classroom-based. 

• These 27 schools had 13,572 
students. 

Figure 5 compares charter schools to 
noncharters in terms of their ability to 
achieve their API growth targets and meet 
the states expectations for improvement. 
After a sharp drop between the first and 
second cycles, charter middle schools 
have shown steady improvement in their 
ability to meet their targets. By compari- 



son, the percentage of successful 
noncharters has fluctuated more. 

The percentage of charter middle 
schools that have reached their targets has 
increased in each of the last three cycles. 
The last two cycles, when the state s chal- 
lenging standards figured predominantly 



in the index, the percentages were an 
impressive 74% and 81%. The extent of 
improvement is also notable. Charter 
middle schools gained an average of 25 
points between the 2003 Base and 2004 
Growth APIs. Noncharters, on the other 
hand, averaged 1 1 points of improvement. 
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figure 5 I Middle schools that made API growth targets 



Two-year API Base/Growth Cycle 





1999/00 


2000/01 


2001/02 


2002/03 


2003/04 


Number of charters in API 
accountability system 


13 


13 


17 


21 


31 


Number of charters with growth data 


11 


12 


16 


19 


27 


Percent of charters with growth data 
making growth targets 


55% 


25% 


50% 


74% 


81% 


Percent of noncharters making 
growth targets 


61% 


51% 


38% 


68 % 


54% 



• It is more meaningful to compare percentages within individual API cycles because of the growing number of charter middle schools 
and the changing index used to measure school performance (the API). 

• The bottom two rows reflect the schools with API data in both years of a given two-year API cycle. 



Data: California Department of Education (CDE) EdSource 5/05 



Charter middle schools shine in similar 
schools rankings in 2004 

An absolute comparison between charter 
and noncharter middle schools is limited 
in its significance because of the small 
number of charters. The 2004 Base API 
statewide rankings include just 42 char- 
ter middle schools. That said, the 



performance of charter middle schools 
based on this indicator presents a more 
positive picture than is the case with 
elementary schools. Once again, the high- 
performing schools were disproportion- 
ately established conversion schools. New 
start-ups were over-represented among 
the lower deciles. 



The 2004 Base API rankings for the 
42 charter middle schools follow: 

• High-performing, deciles 8—10: 

38% (16 schools) 

• Mid-performing, deciles 4—7: 

38% (16 schools) 

• Low-performing, deciles 1—3: 

24% (10 schools) 

When compared to schools with simi- 
lar students and similarly credentialed 
teachers, charter middle schools also 
performed well. In the similar schools rank- 
ings, where each school is compared to 100 
schools with similar challenge levels, 53% 
of schools did quite well. Start-ups, espe- 
cially those that had been open for at least 
two years, were particularly noteworthy. The 
32 middle schools that received similar 
schools rankings performed as follows: 

• High-performing compared to similar 
schools, ranks 8—10: 53% (17 schools) 

• Mid-performing compared to similar 
schools, ranks 4—7: 31% (10 schools) 

• Low-performing compared to similar 
schools, ranks 1—3: 16% (5 schools) 



Algebra I and California High School Exit Exam (CAHSEE) test scores are important indicators 



API scores do not tell the whole story of academic Charter 10th graders did not fare as well on the 
performance. CAHSEE in 2004 

„ . .... . . . . . „ In the 2003-04 school year, 10th graders in nonchar- 

Charter middle schools are not doing as well . . , , 

ter high schools outperformed their counterparts in 

eac mg age ra charter high schools on the exit exam. School-level data 

In the late 1990s, the state adopted math stan- ..... , ...... , 

.... „ , , . , ....... regarding students who passed both the math and 

dards that called for eighth graders to take Algebra _ .. ,, ..... . 

, , ... u , u . , • English language arts portions of the test are not avail- 

1. As a result, middle schools have faced growing ,7 . . ,... , , . 

... . .... . ....... . able. On the math section alone, 74% of noncharter 

pressure to instruct their students in this subject. ... . ... .... , , 

„ ... . , , . . , , . : students passed while 61% of charter students passed. 

Both charter and noncharter middle schools have _ 

l, „n„, . • ... , „ . , The difference in passing rates in English language arts 

roughly 40% of eighth graders enrolled in algebra , . * 6 , 

, . . “ ..... n . was much smaller: 75% of noncharter 10th graders 

and taking the California standards test. But on an , ... ^ . . . ... 

, . ... passed, while 71% of charter students did. 

absolute comparison, the charter students 

performed worse than their counterparts in 2004. As with meeting API growth targets, performance 


Percent of charter and noncharter 10th 
graders who passed the CAHSEE in 2004 




Percent Passing 




Math 


English 


Noncharters 


74% 


75% 


All Charters 


61% 


71% 


Comparisons Among Types of Charters 


Conversion 


80% 


84% 


Start-up 


56% 


67% 


In charter schools, 21% of eighth-grade algebra among different categories of charters also varied. 


Classroom-based 


69% 


76% 


students scored “proficient” or “advanced." A As the table indicates, conversion schools beat 
substantially higher 35% did so in regular public start-ups and classroom-based charters outper- 

schools. It is likely that neither charters nor regular formed their nonclassroom-based counterparts by a 


Nonclassroom-based 


52% 


65% 


Established 


62% 


73% 


public schools are entirely satisfied with their healthy margin in both cases. Established charters 
performance, but the noncharters definitely outper- had slightly higher percentages of students passing 


New 


58% 


67% 


form charters in this key area. the CAHSEE than did new charters. 
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Charter high school performance is mixed 



Key Characteristics of California High Schoolst in 2003-04 

Statewide average: shows the percentage of high school students or teachers in each category statewide. 

Median: shows the percentage for a “typical” school-one at the 50th percentile for a given characteristic. Equal numbers of schools 
have higher and lower percentages. 

Range for middle half: shows the percentages for the schools at the 25th and 75th percentiles. It provides a sense of how schools 
within each group vary and how charters and noncharters differ as well. 





Charter 

Schools (n=151) 


Noncharter 
Schools (n=l,167) 


Student Characteristics 


Median 


Median 


(Statewide Average) 


Range for Middle Half 


Range for Middle Half 


English Learners (15%) 


7%** 


11%* 




2%-20% 


4%-22% 


Hispanic (34%) 


24% 


28% 




9%-48% 


13%-52% 


White (44%) 


46% 


44% 




13%-72% 


19%-70% 


African American (8%) 


6% 


3% 




1%-17% 


l%-9% 


Asian (7%) 


1% 


3% 




0%-3% 


l%-9% 


Free/Reduced Price Meals (28%) 


16%* 


20% * 




0%-49% 


7%-46% 


Parents Not High School Grads (18%) 


10%** 


14%* 




3%-23% 


5%-26% 


One+ Parent a College Grad (22%) 


22%** 


21%* 




16%-34% 


14%-30% 


Student Mobility (22%) 


48%** 


11%* 




20%-66% 


7%-16% 


School Characteristics 


Enrollment (average size 1,347) 


181 


1,501 




101-413 


306-2,294 


Teachers Not Fully Credentialed (12%) 


40% 


7% 




17%-60% 


1%-15% 


Teachers in First Two Years (14%) 


33% 


10% 




20%-50% 


5%-16% 



Data: California Department of Education (CDE)— CBEDS EdSource 5/05 

flNCLUDES ONLY SCHOOLS IN THE API SYSTEM NOT THOSE IN THE ALTERNATIVE SCHOOLS ACCOUNTABILITY MODEL. 

'Data missing for up to 20% of schools. 

**Data missing for at least 20% OF SCHOOLS. 



As was true at the elementary and 
middle-grade levels, the students in char- 
ter high schools tended to be more 
advantaged than those in regular high 
schools, but the differences were much 
less pronounced. When it came to the 
teaching force, however, charter high 
school teachers tended to lack credentials 
and experience by even wider margins 
than was true at the other two levels. The 
charter schools were also dramatically 
smaller. The high mobility rate, measured 
by the percentage of students who first 
attended the school in the present year 
(students in the school’s lowest grade are 
excluded from the calculation), may 
reflect how many new charter high 
schools came into operation during the 
2003—04 school year. 

Charter high schools do well meeting API 
growth targets 

For the 2003 Base/2004 Growth 
API cycle: 

• 120 charter high schools were in the 
API system of accountability. 

• 71 schools (59%) had the data neces- 
sary to determine whether they met 
their growth targets, compared to just 
10 schools in the 1999/2000 cycle. 

• Of those 71 schools, eight were 
conversions and 63 were start-ups; 46 
were classroom-based and 25 were 
nonclassroom-based. 

• These 71 schools had 29,032 
students. 

The number of charter high schools 
in the state’s main accountability system 
has nearly quintupled since it began in 
1999. From the 25 schools that were in 
the system in the first year, the number 
has increased to 120 in 2004. Of those 
schools that were open in 1999, 19 were 
still operating as charters in 2004. But a 
large group of newer schools has entered 
the scene in recent years. As a result, any 



comparison between charter high 
schools in the two time periods is less an 
indication of improvement than a snap- 
shot of two very different populations’ 
performance. 

Figure 6 indicates the number of char- 
ters and noncharters that have met their 



API growth targets over time (see the API 
box on page 4 for further explanation). 
Except in the 2002/2003 cycle — when 
charters did not do quite as well as tradi- 
tional high schools — the charter schools 
with API data have outperformed 
noncharters by healthy margins. In the 
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figure 6 1 High schools that made API growth targets 



Two-year API Base/Growth Cycle 





1999/00 


2000/01 


2001/02 


2002/03 


2003/04 


Number of charters in API 
accountability system 


25 


19 


20 


34 


120 


Number of charters with growth data 


10 


13 


17 


25 


71 


Percent of charters with growth data 
making growth targets 


60% 


38% 


41% 


64% 


58% 


Percent of noncharters making 
growth targets 


41% 


27% 


29% 


67% 


49% 



• It is more meaningful to compare percentages within individual API cycles because of the growing number of charter high schools 
and the changing index used to measure school performance (the API). 

• The bottom two rows reflect the schools with API data in both years of a given two-year API cycle. 



Data: California Department of Education (CDE) EdSource 5/05 



2003/2004 cycle, 58% of charter high 
schools met their growth targets compared 
to 49% of traditional high schools. 

It is also notable that charter high 
schools were able to post an average gain 
of 17 points in this API cycle, which 
compared favorably to noncharters’ aver- 
age of 13 points. One factor that could 
explain some of this difference is that, at 
least in the 2003 state rankings, 20% of 
charter high schools were in the bottom 
decile. Those charters in particular had 
quite a bit of room to grow, achieving an 
average of 35 points of growth in the 
2004 Growth API. 

Absolute comparisons based on 2004 
Base API ranks paint a negative picture 
for high schools 

In the 2004 Base API, 124 charter high 
schools achieved the following rankings: 

• High-performing, deciles 8—10: 

22% (27 schools) 

• Mid-performing, deciles 4—7: 

27% (33 schools) 

• Low-performing, deciles 1—3: 

52% (64 schools) 

Unfortunately, 27 charter high 
schools were missing data and are left out 
of any analysis. 

Start-ups formed the vast majority of 
the 124 schools, and they were spread 



among the decile ranges. The nine conver- 
sion schools with decile ranks, however, 
were over-represented among the higher 
deciles. Another noticeable trend was that 
the 45 nonclassroom-based schools were 
generally found in the lower deciles, with 
only four schools in the upper range. 
Also, an additional 19 nonclassroom- 
based schools did not have rankings. 
Finally, the bulk of the 27 charter high 
schools in the high-performing category 
(deciles 8—10) were start-up schools that 
provide classroom-based instruction. 

Charter high schools tended to be 
at the extremes In the 2004 similar 
schools rankings 

Only 76 charters were included in the 
rankings — meaning that they had more 
than 100 test scores. On this measure, 
charters tended to be at the extremes, 
doing quite well or not well, with rela- 
tively few in between. 

• High-performing compared to similar 
schools, ranks 8—10: 

42% (32 schools) 

• Mid-performing compared to similar 
schools, ranks 4—7: 

16% (12 schools) 

• Low-performing compared to similar 
schools, ranks 1—3: 

42% (32 schools) 



As was true with statewide ranks, 
nonclassroom-based schools tended to earn 
low marks, with 19 of 31 schools in the 
bottom 10%. (Here it is important to 
remember that this type of charter often 
serves students who are there because of a 
lack of fit with mainstream schools and that 
nonclassroom-based charters have quite 
high mobility rates.) In contrast, classroom- 
based charter high schools generally did well 
compared to similar schools, with more 
than 60% reaching the higher decile range. 
Newer schools also tended to rank high 
among comparable schools. 

In 2005 charter renewal requires a new 
minimum performance standard 

California’s initial charter legislation 
included the specific expectation that 
charter schools would be accountable for 
student performance when they applied 
for charter renewal every five years. At the 
time, however, the state did not have the 
comprehensive testing and accountability 
systems it has since developed. The defi- 
nition of acceptable student performance 
was left to school organizers and their 
chartering districts to determine. This 
changed in 2003 when lawmakers set 
minimum expectations for charter school 
performance based on the API. 

Beginning in January 2005, a charter 
school requesting renewal must either: 

• meet its API growth targets (in the 
prior year, in two of the last three 
years, or in the aggregate for the prior 
three years); or 

• rank in at least Decile 4 on its Base 
API (using either state or similar 
schools rankings in the prior year or in 
two of the last three years); or 

• qualify for the Alternative Schools 
Accountability Model. 

Between January and August 2005, 
the 80 charter schools up for renewal will 
be the first to be affected by the new law. 
Among those schools, 78% have satisfied 
at least one of the renewal criteria. The 
remaining 18 schools lack the relevant 
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data. Under Assembly Bill (AB) 1137, if 
a charter school cannot show that it met 
one of the performance criteria described 
above, the charter-granting authority 
must determine that the academic 
performance of the charter school is at 
least equal to that of the public schools 
the charter school pupils would otherwise 
have attended and the schools in the local 
district that have similar racial composi- 
tions. The California Department of 
Education reviews the chartering agency’s 
analysis and makes a recommendation to 
the agency regarding the school’s renewal. 

Another 72 charter schools will be up 
for renewal between September 2005 and 
August 2006. Based just on their API 
information from 2002 to 2004, 71% 
have already qualified for renewal. Eight 
schools have not yet met the criteria, and 
13 lack the relevant data. 



Research findings conflict in charter/ 
noncharter comparisons 



A number of state and national studies 
have attempted to address whether the 
academic performance of students in 
charter schools is better, worse, or 
comparable to those who attend nonchar- 
ter schools. Invariably proponents and 
opponents of charters gather around each 
new study, mining it for evidence to 
support their arguments. All of these 
studies, however, are subject to criticism 
because of the many challenges previously 
described in this report. They also all use 
different methodologies. It may not be 
surprising, then, that they all seem to 
come to somewhat different conclusions. 

National studies are particularly complex 

National studies face a particularly diffi- 
cult challenge because researchers must 
first somehow choose among the varied 
types of test data available. 

Much of the national research has 
depended on the National Assessment of 
Educational Progress (NAEP) as its 



measure of performance. NAEP is the 
only assessment taken by students in 
every state, but it has certain limitations 
because only a portion of students take it 
and each student takes only a portion of 
the test items for a given subject. A 2004 
report by the American Federation of 
Teachers (AFT) generated headlines and 
subsequent controversy by claiming that 
fourth and eighth grade students in char- 
ter schools were scoring lower on NAEP 
than their counterparts in noncharter 
schools. Entitled Charter School Achievement 
on the 2003 National Assessment of Educational 
Progress, the report compared NAEP 
reading and math scores in a nationally 
representative sample of charter and 
noncharter public schools. Critics say the 
report looked at schools serving just 3% 
of charter school students nationwide. 
(See page 21 for more about this study.) 

Other researchers have used state test- 
ing system results as their measure of 
performance. One such study pronounced 
that charters were doing better than regu- 
lar schools. Caroline Hoxby of Harvard 
University and the National Bureau of 
Economic Research authored A Straightfor- 
ward Comparison of Charter Schools and Regular 
Public Schools in the United States in 2004, on 
the heels of the AFT report. She 
compared the reading and mathematics 
proficiency of elementary charter school 
students to noncharter students based on 
the specific tests given in their respective 
states. She found that charter schools 
outscored the nearest noncharter public 
schools. Hoxby also looked specifically at 
California schools. (See page 21 for more 
about this study.) 

Again, these two national studies — and 
many others — have both supporters and 
detractors. The same is true of the Califor- 
nia-based studies that have been done. 

California studies differ in their methodologies 
and their conclusions 

Researchers who have compared the 
performance of charters to noncharters 



in California have also used different data 
sets and analytical methods, and they have 
reported different results. Studies consid- 
ering only API scores as issued by the 
state concluded that charter schools 
performed somewhat better. However, 
researchers with access to student-level — 
as opposed to only school-level 
scores — found that regular public 
schools did slightly better, though not 
always by a statistically significant margin. 

One of the most substantial of these 
studies was done by RAND and overseen 
by the Legislative Analyst’s Office. This 
July 2003 study, Charter School Operations 
and Performance: Evidence from California, 
compared the performance of charters 
and noncharters and also looked at how 
performance varies among different kinds 
of charter schools. The authors also did 
analyses using both the API and SAT-9 
(Stanford Achievement Test, 9th Edition) 
test scores. Their conclusions were slightly 
mixed with respect to comparisons 
between charters and regular schools and 
among various types of charters. (See page 
22 for more about this study.) 

David Rogosa, an education professor 
and statistician at Stanford University, 
has been vocal in charter school research, 
sometimes critiquing other researchers’ 
work. In 2003 Rogosa used his unusual 
expertise on the API to develop and 
publish an analysis that depended on 
student-level data to simulate API scores 
for the years 1999 to 2002. These mock 
APIs were constructed entirely with 
scores from the nationally normed SAT- 
9. His analysis showed that the 
performance of charter school students 
varied depending on students’ grade level 
and socioeconomic status. (See page 23 
for more about this study.) 

These California studies — and 
others that have been done — primarily 
reflect performance on the SAT-9, the 
state’s basic skills test used from 1998 
through 2002. To a great degree, they 
pre-date the California standards tests 
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California standards tests provide a reliable though imperfect 
basis for comparison 



Evaluations of academic performance would ideally consider more than just standardized test scores. 
However, in California the STAR tests are the only academic measures with the qualities necessary to 
make a valid comparison between schools, including the comparison of charters and noncharters. 

Using STAR tests to compare schools has several advantages 

• All schools arguably have access to the material covered on the STAR tests. 

• The assessments have been screened for ethnic bias, for being a reasonable gauge of what they are 
purported to measure, and for being reliable-meaning that if two highly similar students took the test, 
they would get similar scores. 

• The tests are secure: teachers and students do not know in advance what the exact questions will be. 

• Virtually all students in both types of schools take the STAR tests. As a result, it is unlikely that, for 
example, only the stronger or weaker students are taking the test in one type of school or the other. 

The STAR program does not directly measure the “value added” by individual schools 

The data only provide annual snapshots of how each group of students perform on a particular test. Given 
the state’s still-developing data system, the performance of last year’s third graders can be compared to 
this year’s third graders, but the progress of one group of third graders over time is not measured. Were 
last year’s third graders to be compared to this year’s fourth graders, other data problems could arise 
related to the connection between third and fourth grade tests and changes in the groups of students 
tested due to student mobility. 



(CSTs), which assess students’ profi- 
ciency on the states adopted content 
standards and which have been incorpo- 
rated into accountability measures more 
recently. This is because the studies were 
done before multiple years of CST data 
were available. The emphasis on SAT-9 
data may make the performance assess- 
ments somewhat less meaningful in 
today’s standards-focused context. They 
also contrast with the general finding 
noted previously that charter school 
performance on the API has improved 
since 2003 when the standards-based 
tests took on more weight in the index. 

EdSource tries another approach 

Along with the controversies, each succes- 
sive analysis directly comparing the 
academic performance of charter and 
noncharter schools has helped to show 
how complicated such comparisons can 



be and how the results can vary. EdSource 
used that growing knowledge base to 
guide yet another attempt to answer the 
most commonly asked question: Are char- 
ter schools more successful less successful, or 
simply comparable to noncharters in their ability 
to improve student peformanceP 

Prior research highlights shortcomings 
of the API 

The earlier studies mentioned above — 
particularly the California studies — 
influenced the way that EdSource 
approached this question. The first 
consideration was to make sure the meas- 
ure of performance looked at 
improvement over time and did so in a 
thoughtful way. Second, the schools 
being compared were selected carefully to 
ensure that they face similar challenges 
with respect to student demographics 
and that their starting points, in terms of 



performance, were similar. Comparing 
charters and noncharters that are compa- 
rable in both respects makes it possible to 
focus more closely on the direct impact of 
charter status on improvement in student 
achievement. 

To meet these criteria, this analysis 
focuses on a small number of well-estab- 
lished charter and noncharter schools and 
on discrete performance measures. As 
such, the findings cannot be used to 
generalize about all areas of student 
performance or all charter schools. In 
particular, it could be argued that it 
distorts charter school achievement by 
focusing only on the more stable charters. 
The 89 charter schools selected serve a 
total of 53,473 students in 2003—04. 

EdSource’s analysis is noteworthy 
because it uses scores on the CSTs rather 
than the state’s API or SAT-9 scores. The 
CST data provide more detailed informa- 
tion than API scores, and they are salient 
in California’s new accountability climate 
under the federal No Child Left Behind 
Act (NCLB). 

The experience of the RAND 
researchers influenced this decision. In its 
evaluation of California charter schools, 
the RAND team used API scores for a 
variety of reasons. They found, however, 
that because the index aggregates all test 
scores in a school into one number, it 
obscures important information. 
Further, the weight given to various 
components of the index is based on a 
policy decision by the State Board of 
Education. For example, in elementary 
schools, English scores count for 60% of 
the API and math scores count for 40%. 
If one wants to know how students in a 
particular grade or subject did — or if 
one disagrees with the board’s weighting 
of various subjects in the index — API 
scores are not satisfactory. 

In addition, the API system divides 
schools into three broad school types — 
elementary, middle, and high. Because 
grade configurations within each type 
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vary, and score growth varies within each 
grade, school-level API scores do not offer 
precise “apples to apples” comparisons. 

As an alternative to the API, RAND 
and other researchers used SAT-9 test 
scores for analyses of charter perform- 
ance. Now, however, schools and districts 
are focused on increasing the percentage 
of students scoring “proficient” or above 
on the CSTs. And multiple years of data 
on these tests are now available. Along 
with new weighting in the API, accounta- 
bility for “adequate yearly progress” under 
NCLB makes these tests more important. 

EdSource comparison uses CST scores 
EdSource compared growth in the 
percent of students scoring proficient or 
above on math and English CSTs in char- 
ters and noncharters between 2002 and 
2004. The analysis includes only scores 
from grades 3, 7, and 10 in English and 
only grades 3 and 7 in math. (Students 
take different math CSTs in grades 8—1 1 
based on the course they are taking. As a 
result, it is not sound to compare schools’ 
scores on math CSTs beyond grade 7. 
The exit exam, a uniform statewide meas- 
ure, is more appropriate. Those scores are 
discussed earlier in this report.) 

NCLB has encouraged California 
schools to focus on increasing the 
percentage of students scoring proficient 
or above on English and math CSTs. 
However, a substantial portion of 
students still score at the lowest profi- 
ciency level, “far below basic.” Helping 
them improve is also an important goal. 
The EdSource analysis therefore also 
considers schools’ ability to decrease the 
percentage of students scoring at this 
lowest performance level. 

The results show strong performance 
among established charter schools 

Using a carefully selected subset of 
schools, EdSource made two compar- 
isons. One compared the amount of 
improvement that a group of charter 
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Creating the EdSource dataset 



To set up a comparison of charter and tradi- 
tional public schools, EdSource combined data 
from three electronic files provided on the Cali- 
fornia Department of Education website: 1) CST 
scores in English language arts and math, 2) 
student demographics, and 3) the “school char- 
acteristics index" (SCI) scores for similar 
schools rankings. The data pertain to the 
2001-02 through 2003-04 school years. 

Schools had to fit specific criteria to be used in 
the analysis, including: 

• Any school that experienced a significant 
demographic change in its student body was 
removed from the analysis. A “significant 
demographic change” is defined as a change 
of more than 10 percentage points in more 
than one of the following categories: English 
learners, students eligible for free/reduced- 
price meals, ethnic groups, and parental 
education levels (e.g., percent of students 
with at least one college-educated parent). 

• Only schools that had been open since at 
least 2000-01 were included. Given that 
many start-up charter schools face cash flow 
and facilities challenges in their early years, 
only start-ups that had been open since 
1999-2000 were included. 

• To measure challenge, EdSource used the 
school characteristics index (SCI). The CDE 
computes the SCI each year as a first step to 
creating the “similar schools" rankings in the 
Base API. Elementary, middle, and high 
schools have different SCIs. Thus an elemen- 
tary school with an SCI of 150 has a different 
challenge level than a middle school with the 
same SCI. Because so many charter schools 
have nontraditional grade configurations 
(e.g., some charters with 3rd graders are 
classified as middle schools), EdSource 
could not simply use charter schools’ SCI 
scores to find comparable noncharter 
schools. Instead, EdSource ranked schools of 
similar type (elementary/middle/high) 
according to their SCI and assigned each 



school a percentile ranking. With those 
percentile rankings, EdSource could compare 
two schools serving a common grade based 
on their rankings, regardless of whether they 
were elementary, middle, or high schools. For 
example, an elementary school at the 30th 
percentile in its SCI rankings and serving 
third graders could be compared to a middle 
school at the 30th percentile in its SCI rank- 
ings and serving third graders. For a 
noncharter to be compared to a charter, its 
SCI percentile ranking had to be within five 
percentile points of the charter’s SCI 
percentile ranking. 

• For two schools to be compared, they also 
had to be similar in their starting points vis- 
a-vis student performance. In 2002, the 
schools were within five percentage points of 
each other in their percentage of students 
scoring proficient and above, or in those 
scoring far below basic. For example, an 
elementary charter school at the 65th 
percentile on the SCI with 35% of its third 
graders scoring proficient and above could 
be compared to any school ranking from the 
60th to 70th percentile on the SCI and with 
30% to 40% of its third graders scoring profi- 
cient and above. 

In the end, 89 charter schools and 5,762 tradi- 
tional public schools met all the criteria. 
Schools serving more than one of the relevant 
three grades— 3, 7, and 10-were used in more 
than one grade-level analysis. For example, if a 
school served grades K-8, its grade 3 scores 
were used in the analysis of grade 3 perform- 
ance, and its grade 7 scores were used in the 
grade 7 performance analysis. A handful of the 
charter schools that performed considerably 
better or worse than schools facing similar chal- 
lenges did not have a comparison group and 
were not included in the school-level match- 
ups. Noncharters that were similar to multiple 
charter schools were assigned to multiple 
comparison groups. 
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figure 7 



Scores for charter school students improved more than those 
for comparable noncharter students from 2002 to 2004 



Average percentage-point change in students scoring.. 
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Data: Adapted 


from California Department of Education (CDE)— STAR research files 


EdSource 5/05 



schools made to the improvement of a 
group of similar noncharter schools. The 
other looked at the percentage of charter 
schools that showed more improvement 
than their comparison schools. 

Charters outperformed noncharters 
in both cases. The first analysis found that 
charter students showed slightly greater 
improvement than those in noncharters 
on the English and math CSTs from 2002 
to 2004. The second analysis showed 
charter schools performing better in a 
majority of direct comparisons between a 
charter school and a group of similar 
noncharters. (For more details on how 
EdSource narrowed the set of schools for 
its analysis, see the box on page 17.) 

Each combination of grade, subject, 
and performance level — for example grade 
3, math, proficient and above — was 
considered separately. For each school, the 
percentage scoring at a particular perform- 
ance level in 2002 was subtracted from the 
2004 percentage. That improvement in percent- 
ages was the important variable for both analyses. 
(“Improvement” was counted two ways: as 
an increase in the percent of students scor- 
ing proficient and above, and as a decrease 
in the percent scoring far below basic.) It 
is important to note the improvement is 
between two different groups of 
students — for example the third graders of 
2002 and the third graders of 2004. 

CST scores for charter students 
improved by greater margins 
In the first computation, each school’s 
improvement level was weighted by the 
number of test-takers. The averages were 
then calculated for the charter and nonchar- 
ter groups.Taking a weighted average creates 
(figuratively speaking) one big charter 
school and one big noncharter school. 

Figure 7 shows the weighted average 
improvement for students in charter 
schools and in the comparison nonchar- 
ter schools in each grade, subject, and 
proficiency level. On all 10 comparisons, 
the students in charter schools improved 



their scores more than noncharters. This 
means that the charters studied were 
better than their comparison schools at 
increasing the percentage of high-scoring 
students and reducing the percentage of 
low-scoring students. Although the 
differences appear small — ranging from 
0.6 percentage points to 3.8 points — 
they are statistically significant, meaning 
that they fall outside the typical range of 
random variation. The smallest gap 
between charters and noncharters was in 
third grade English for high-scoring 
students. Both groups actually declined in 
that percentage from 2002 to 2004, but 
charters did so by slightly less (- 1 .8 vs. -2.4 
percentage points). 

The greatest gap was also in third 
grade, but on math scores. There, charters 
decreased the far below basic group by 7 
percentage points versus 3.2 points for 
noncharters. The differences in improve- 
ment in IOth-grade English scores were 
also relatively large. 

Even the largest differences between 
charters and noncharters, however, are 
not as large as the variation within each 
type of school. For example, in grade 7, 
English, proficient and above, one charter 
school declined by 16 percentage points 
while another improved by 29 points. 
That 45-point difference within the char- 
ter group compares to a difference of less 



than one percentage point between charters 
and comparison noncharters. 

School-level comparisons also show 
greater improvement among charters 
For the second analysis, EdSource calcu- 
lated the percentage of charters that 
showed more improvement than their 
comparison schools did. Each charter 
school’s improvement was compared to 
the weighted average improvement of its 
noncharter comparison schools. 

Figure 8 shows the number and 
percent of charter schools that outper- 
formed their comparison schools. 
Charter schools generally did better than 
their comparison groups. Again, the 
comparisons look both at schools where 
more students improved into the “profi- 
cient and advanced” performance bands, 
and at those where more students 
improved out of the “far below basic” 
band. In eight of the 10 separate compar- 
isons — based on grade levels, subjects, 
and performance bands — a majority of 
charters outperformed their noncharter 
comparison group. 

The comparison in which charters 
were least successful, with just 43% 
“winning,” was in Grade 3 math in the 
proficient-and-above performance band. 
On the other hand, 73% of charter 
schools outperformed their comparison 






o 



18 • How Are California’s Charter Schools Performing? • May 2005 



EDSOURCE REPORT 









figure 8 Number and percentage of charter schools that improved more 
than their comparison schools 



Change in percentage of students scoring at specified levels on California standards tests 
from 2002 to 2004 
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schools in decreasing the percentage of 
7th graders scoring in the far-below-basic 
band on the math CST. 

The results also reinforced a familiar 
pattern in charter school achievement: 
conversion schools outperformed start- 
ups (with seventh-grade English the one 
exception). Consider the example of 
third-grade English, in which 45% of 
charters prevailed based on the percent- 
age of students scoring proficient and 
above. Among conversion schools, 50% 
outperformed their comparison schools 
while 32% of start-ups beat their 
comparison schools. 

This analysis also varied in an impor- 
tant way from the API comparisons 
provided earlier in this report. Class- 
room-based charters did not always 
outperform their nonclassroom-based 
counterparts. For example, in 7th grade 
English, 10th grade English, and 7th 
grade math, nonclassroom-based schools 
outperformed their comparison nonchar- 
ter schools more often than the 
classroom-based charters beat out theirs. 

On the other hand, subdividing 
charter schools by type revealed that 
charters that were both conversion and 



classroom-based fared better than 
schools that were both start-up and 
nonclassroom-based. (Schools are not 
subdivided into “established” and 
“new” in this analysis because all schools 
included in the dataset met EdSource’s 
definition of “established.”) 



EdSource finds noteworthy patterns, 
but more study is needed 



Many studies of charter school perform- 
ance focus on the viability of the charter 
approach. Often, the explicit or implied 
conclusion is that this educational exper- 
iment either should or should not 
continue. But as the analyses presented in 
this report make clear, there is not a 
preponderance of evidence that makes an 
“up or down” vote on the charter school 
concept obvious or even desirable. 

More than 500 charter schools are in 
operation today in California and some 
of them have been open for more than a 
decade. It behooves educators, policy- 
makers and the public to learn more 
about the schools themselves, their 
successes, and their failures. There are also 
many questions to be explored in regard 



to the best role they can play in the public 
education system and what conditions 
make them most successful. 

Key findings provide a fresh and revealing 
look at charter performance 

Using state testing data, EdSource found 
several patterns of charter school 
performance that are particularly note- 
worthy. Collectively they present some 
reasons for optimism and some causes for 
concern. They also raise a number of 
interesting questions that should be 
explored more thoroughly. 

Missing test data from a large percent- 
age of the states charter schools undermines 
the ability to assess their academic perform- 
ance and thus their effectiveness as an 
alternative learning environment for some 
students. For the 2004 Growth Academic 
Performance Index, about 23% of the char- 
ter schools in the states API system lacked 
sufficient test data to even receive API 
scores. That compares to just 6% of the 
noncharter schools in the API system. Only 
13% of classroom-based charter schools 
were missing data, while 46% of nonclass- 
room-based charter schools did not 
administer the states standards based tests 
to enough students to yield official API 
results. (See page 6.) 

Among charter schools with data, 
particularly classroom-based charters, 
improvement meeting academic growth 
targets is particularly notable. The relative 
performance of charters started to 
improve in 2003 when API scores began 
emphasizing the California standards tests 
instead of off-the-shelf norm-referenced 
tests. In 2004, 64% of classroom-based 
charters (which constitutes 60% of all 
charters) met their 2004 API growth 
targets, outperforming both noncharter 
schools (48%) and the nonclassroom- 
based charter schools (44%) for which 
data is available. (See page 8.) 

Elementary charters lag behind non- 
charters on Base API scores, but have 
recently done better meeting growth 



o 



May 2005 • How Are California’s Charter Schools Performing? • 19 



© Copyright 2005 by EdSource, Inc. 





Copyright 2005 by EdSource, Inc. 



EDSOURCE REPORT 






targets. In an absolute comparison between 
all elementary charters and noncharters, 
the charter schools on average have slightly 
lower 2004 Base API scores. This is in 
spite of the fact that elementary charter 
schools serve a smaller proportion of 
English learners and low-income students 
than their noncharter counterparts. 
However, charter elementary schools have 
improved the rate at which they meet their 
API growth targets. In 2004 the group of 
elementary charters was the first to outper- 
form noncharters, with 57% of them 
meeting their targets compared to 46% of 
noncharters. (See page 9.) 

The small group of charter middle 
schools performed quite well. Just 3 1 char- 
ter middle schools were included in the 
states accountability system in 
2003/2004 and that small number 
tempers any assertions about the academic 
performance of charters serving this age 
group. That said, among the 27 of those 
charter middle schools that had 2004 
Growth API scores, 81% met their growth 
targets (compared to 54% of noncharter 
middle schools). (See page II.) 

California’s middle grades charter 
schools on average serve fewer disadvan- 
taged students than their noncharter peer 
schools. While it is therefore not surpris- 
ing that they ranked relatively high on 
their 2004 Base API scores, they also 
compared favorably to noncharter middle 
schools with similar students based on 
the states similar schools rankings. 

Firm conclusions about charter high 
school achievement are problematic. 
Charter high schools increased in number 
very quickly, going from 34 schools with 
API growth data in 2003 to 120 in 2004. 
Most of these new charters are start-up 
schools that tend to differ substantially 
from noncharters in the students that 
they serve and the teachers that they 
employ. (See page 13.) 

More than half of charter high 
schools ranked in Deciles 1—3 (the 
bottom 30%) based on the 2004 Base 



API. But comparing them to noncharters 
serving similar students nets more mixed 
results, with equal proportions comparing 
well and poorly. In terms of improvement, 
missing data is again a challenge. Just 71 
high schools had growth data for the 
2003/2004 cycle. Of those schools, 
58% met their growth targets compared 
to 49% of regular high schools. 

In an EdSource study that matched 97 
well-established charters against a large 
group of comparable noncharters, the 
charter schools’ test scores generally 
improved more than those of noncharters. 
Based on an analysis of California stan- 
dards tests (CSTs) for grades 3, 7, and 10 
in English and grades 3 and 7 in math, 
charters did slightly better at increasing 
the percentages of students scoring 
“proficient and above” and decreasing the 
percentage scoring “far below basic” from 
2002 to 2004. The differences appear 
slight, but they are statistically signifi- 
cant — meaning outside the normal range 
of random variation. (See page 16.) 

In eight out of 10 sets of school-level 
comparisons between charters and 
noncharters, a majority of charters 
showed more improvement on the math 
and English CSTs from 2002 to 2004 
than their comparison noncharter schools: 

• Charters serving grade 3 students 
outperformed their comparison 
noncharter schools in improving their 
percentages scoring “far below basic” but 
not those scoring “proficient and above.” 

• A majority of charters serving grade 7 
students outperformed their compari- 
son schools in both subjects and at 
both score levels. 

• In 1 0th grade English, a majority of 
charters showed greater improvement 
than their matched noncharters at 
both score levels. 

Continued study of California charter 
school performance is important 

State and federal policymakers are eager 
to find solutions for the state’s chroni- 



cally low-performing schools. Under the 
No Child Left Behind Act, turning 
them into charter schools is one option, 
but is conversion to charter status an 
effective solution? 

Not necessarily, but it might be 
under certain circumstances. Just as in 
the regular public school system, the 
charter community has examples of 
both low-performing and exemplary 
schools. Before state and local policy- 
makers advocate the use of public tax 
dollars to convert low-performing 
public schools to charter status, they 
should closely examine the types of 
charter schools that are succeeding, for 
what kinds of students, under what 
conditions. 

That said there are real success stories 
among charter schools. Many of them are 
providing exciting learning environments. 
And many are successful serving students 
who for various reasons have not found a 
good fit in a regular public school setting, 
including many low-income children of 
color. Of particular note is the burgeon- 
ing use of charters as a strategy for 
addressing the stubborn challenges of 
high school reform, particularly in disad- 
vantaged communities. 

In addition — based on state testing 
data — charter schools have recently 
started to make impressive gains, improv- 
ing performance and meeting academic 
growth targets faster than noncharters in 
many cases. The charter API growth data 
for 2004 is definitely promising, but one 
year does not yet make a trend. 

California’s charter school experi- 
ment can be a lightning rod for 
conflicting political views. But students 
and the state would be better served if 
charter schools were instead seen as 
laboratories of school improvement that 
add to the information researchers, 
educators, and policymakers can use to 
make sure that every child has an oppor- 
tunity for academic success in the state’s 
public schools. QE 
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National and California studies of charter school performance 



CONTROVERSIAL REPORT CLAIMS CHARTERS NATIONALLY 
FALL SHORT OF EXPECTATIONS 

In 2004, in a national report titled Charter School Achievement on the 2003 
National Assessment of Educational Progress (NAEP), the American Federation 
of Teachers (AFT) compared NAEP reading and math scores in a nationally repre- 
sentative sample of charter and noncharter public schools. The paper reportedly 
looked at schools serving about 3% of charter school students nationwide. 

The NAEP reports include the percent of students scoring at four proficiency 
levels and an average “scale score’’ that reflects the difficulty of the questions 
answered correctly. In each state, only a sample of students take the test, and 
students take different portions of the test. 

Charter students in grades 4 and 8 score below noncharter students on NAEP 

AFT found that fourth graders in charter schools scored an average of six points 
lower on the scale (228 vs. 234) in math and seven points lower (210 vs. 217) 
in reading. Charter students in eighth grade scored five points lower in math 
and two points lower in reading. AFT stated that, with the exception of eighth 
grade reading, the results were statistically significant. (Statistical significance 
is reached when differences are outside the typical range of random variation.) 
AFT estimated that the differences between charter and noncharter perform- 
ance translate to about half a school year. 

AFT also disaggregated the data by poverty status, locale, and ethnicity, finding 
similar differences for low-income students. However, when students were 
compared only to those of the same ethnicity, or from similar locales, the differ- 
ences were much less. An exception was in central cities, where fourth grade 
charter students trailed by seven points in math. 

A group of researchers questions the AFT analysis 

A group of researchers from across the country faulted the report for relying on 
limited student background information and considering only one background 
factor at a time rather than considering all available factors simultaneously. 
(For example, focusing on race does not take into account differences in aver- 
age income among races. Looking at all background factors together allows a 
researcher to isolate the impact of each factor.) In addition, they criticized the 
report for ignoring important student background information and for only 
examining test scores for one point in time. 

For a copy of the AFT report, go to www.aft.org/pubs-reports/downloads/ 
teachers/NAEPCharterSchoolReport.pdf. It includes a subset of data for Cali- 
fornia students. 

2003 PILOT STUDY USING NAEP RESULTS ALSO AVAILABLE 

This study looked at fourth grade math and English performance at 
150 randomly selected charter schools. For a copy go to: 

http://nces.ed.gov/nationsreportcard/pdf/studies/2005456.pdf 



ECONOMIST COMPARES WITHIN STATES, FINDS MORE POSITIVE RESULTS, 
AND ALSO RECEIVES CRITICISM 

In a study of charter school performance entitled A Straightforward Compari- 
son of Charter Schools and Regular Public Schools in the United States, 
Harvard economist Caroline Hoxby compared the reading and mathematics 
proficiency of charter school students to noncharter students based on state- 
specific tests. She generally dealt with fourth graders' scores but used grades 
3 or 5 in cases where grade 4 data were unavailable. 

The charter schools were compared to the schools that their students would 
most likely otherwise attend-the nearest noncharter school-and also to the 
nearest noncharter with a “similar racial composition.’’ In 92% of the cases, the 
noncharter that had the similar racial composition was also the closest school. 
Hoxby’s report states that she covered about 99% of the nation’s fourth grade 
charter school students. (Presumably she meant 99% of charter students who 
were tested, not all students.) 

Fourth grade score comparisons yield good news for charters 

Based on both reading and math test scores, Hoxby found that charter schools 
outscored the nearest noncharter public schools. The percent of students profi- 
cient in reading was four percentage points higher, and in math the advantage 
was two points. When compared to the nearest regular public school with a 
demographically similar student body, charter schools showed a five percent- 
age point advantage in reading and a three point lead in math. 

Hoxby reported data for those states with statistically meaningful populations 
of charter school students, generally more than 200. That included California. 

To get a copy of Hoxby’s study, including the Calfornia results, go to: 

www.wacharterschools.org/learn/studies/HoxbyCharters_Dec2004.pdf 

Hoxby’s methods have been questioned 

In a paper titled Advantage None: Re-Examining Hoxby's Finding of Charter 
School Benefits, the Economic Policy Institute (EPI) asserts that the professor’s 
analysis does not adequately control for student backgrounds. When EPi 
analyzed Hoxby’s dataset and controlled for race and income, it found only one 
state-California-where reading scores were higher to a statistically significant 
degree in charters as opposed to noncharters. It found no states where char- 
ters had a statistically significant edge in math. 

To get the EPI critique, go to: http://epinet.org/content.cfm/bpl58 

SRI INTERNATIONAL REPORT LOOKS AT FIVE OTHER STATES 

This analysis of charter schools’ ability to meet performance standards in five states— 
Texas, Colorado, Illinois, Massachusetts, and North Carolina-is available at: 

www.sri.com:8000/policy/cep/choice/PCSP_FinalReport_2004_0PA_ 

approved.pdf 
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RAND STUDY LOOKS AT BOTH API AND SAT-9 SCORES IN CALIFORNIA, 

WITH MIXED RESULTS 

Called for in legislation and overseen by the Legislative Analyst’s Office (LAO), 
the RAND report— Charier School Operations and Performance: Evidence from 
California- compared the performance of charters and noncharters and looked 
at how performance varies among different kinds of charter schools. The latter 
analysis distinguished between charter schools based on the way they were 
founded (as a conversion of an existing public school or a wholly new start-up), 
whether their instruction follows a traditional classroom-based approach or 
not, and whether the school had been operating under a charter for at least 
three years. 

Based on API, charters and noncharters show no significant differences 
in student performance 

After adjusting for changes in demographics over time in each school, RAND 
compared annual growth in API scores among charters and conventional 
schools in the 1999-2000 through 2001-02 API growth cycles.The study found 
no statistically significant difference between charters and noncharters. The 
type of charter school did not matter either. 

Analysis of SAT-9 test scores showed slightly better performance 
for noncharters 

RAND did a separate comparison using individual students’ test scores, adjust- 
ing for differences in ethnicity, parent education, English learner status, gender, 
and whether a given year was the student’s first at the school. All of these 
student characteristics are correlated with student performance. Controlling for 
them is a way for researchers to attempt to isolate the effect the school per se 
has on student performance. 

With these adjustments, they found some statistically significant differences in 
the average percentile scores of students in charter versus noncharter schools. 
For example: 

• In math: noncharters scored higher in the elementary (49.3 vs. 47.8) and 
secondary (47.7 vs. 45.4) levels. (“Secondary” schools were defined as 
middle and high schools.) 

• In reading: noncharters scored higher at the secondary (42.1 vs. 40.7) level, 
but scores at the elementary level were nearly identical. 

But classroom-based charter schools outperform noncharters 

RAND also looked at performance differences based on the type of charter school. 
Classroom-based charters generally scored slightly higher than regular public 
schools. However, the average student in schools offering nonclassroom-based 
instruction had scores that were between five and nine points lower. Clearly, 
nonclassroom-based schools bring down the average scores of charter schools. 

In discussing this finding, the RAND report notes that nonclassroom-based 
schools may differ from others in ways not easily captured in statistics. For 
example, if nonclassroom-based schools are designed to help students who 
have left conventional public schools because of problems they have in tradi- 
tional settings, then it is not appropriate to compare nonclassroom-based 
charters to conventional schools. Such differences in students would not show 
up in statistical data about whom a given school serves. 



RAND also found differences between the performance of new and estab- 
lished charters. One interesting finding was that nonclassroom-based charter 
schools differed depending on whether they were less than three years old. 
The newer nonclassroom-based charter schools performed substantially 
better-from four to seven percentile points higher on the SAT-9 depending 
on the grade and subject. It is unclear, however, what accounted for the 
difference. 

To see the RAND report, go to: www.rand.org/publications/MR/MR1700/. 
For the official analysis and report to the Legislature by the LAO, go to: 

www.lao.ca.gov/2004/charter_schools/012004_charter_schools.pdf 



STUDY BY RESEARCHERS AT CAL STATE L.A. IS GOOD NEWS 
FOR CALIFORNIA CHARTERS, BUT RESULTS ARE QUESTIONED 

Three researchers from California State University, Los Angeles (CSULA)— 
Simeon Slovacek, Antony Kunnan, and Hae-Jin Kim-used API data to conclude 
that charter schools did a better job of improving the performance of low- 
income students than regular public schools did. 

In their March 2002 report -California Charter Schools Serving Low-SES 
Students: An Analysis of the Academic Performance Index- the researchers 
used Base API scores from 1999 through 2001 to compare the performance 
of charters and noncharters with a majority of students eligible for 
free/reduced-price meals. Their analysis of the dataset they created 
revealed that, in schools with at least 50% low-income students, the API 
scores of charters went up by 22.6% from 1999 to 2001 while the scores of 
noncharters improved 19.4%. The difference was more pronounced in 
schools with at least 75% low-income students— 28.1% improvement in 
charter schools versus 23.8% for noncharters. The CSULA report also 
concluded that the type of instruction that a charter offered, its length of 
time open, and whether it was a start-up or conversion did not seem to make 
a difference in performance. 

This analysis was criticized based on the quality of its dataset and the fact that 
it lumped all the charter schools together as one group without regard to 
whether the percentages of elementary, middle, and high schools are the same 
as in the universe of noncharter schools. 

To see a copy of this study go to: 

www.calstatela.edu/academic/ccoe/c_perc/rptl.pdf 

FOR ADDITIONAL STUDIES OF CALIFORNIA CHARTER SCHOOL ACADEMIC 
PERFORMANCE, SEE: 

• The Performance of California Charter Schools by Margaret E. Raymond of 
CREDO at the Hoover Institution at Stanford University. Go to: 

http://credo.stanford.edu/downloads/ca_chart_sch.pdf 

• Catching the Wave: Lessons from California’s Charter Schools by Nelson 
Smith of the Progressive Policy Institute. Go to: 

www.ppionline.org/documents/CA_Charters_0703.pdf 
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ROGOSA’S STUDENT-LEVEL ANALYSIS SHOWS MIXED RESULTS 
IN CALIFORNIA THROUGH 2002 

In an analysis published in June 2003, Stanford Professor David Rogosa worked 
with Academic Performance Index (API) scores to compare student perform- 
ance in charter and noncharter schools. An expert on the API calculation and 
data, Rogosa developed simulated API scores for the years 1999 to 2002.These 
mock APIs were constructed entirely with scores from the nationally normed 
SAT-9 (Stanford Achievement Test, 9th Edition). 

The dataset consisted of student test scores from 93 charter schools that 
received API scores from 1999 through 2002 and from 6,584 noncharters. (A 
small percentage of each type of school was missing one year’s API score.) 

Charter students in most grades show less improvement than those in 
noncharters 

For his comparisons, Rogosa grouped the test scores from all charter students 
across the state by grade and computed API scores. For example, he grouped 
the scores of all second grade students from all of California's charter schools 
and computed an API score as if they attended one large charter school. He did 
so to avoid comparing, for example, charter schools with grades 3-6 to regular 
schools with grades 2-5 because different grades have shown different levels 
of score-growth. Rogosa was able to create these mock API scores because he 
had access to student-level data. 

Rogosa showed results that varied by grade level and by students’ socioeco- 
nomic status. As the table below shows, at four grade levels (2, 4, 10, 11) 
charter students showed greater improvement (3-11 API points) from 1999 to 
2002, but in the other six grades, noncharter students demonstrated better 
gains (2-33 points). The number of charter schools contributing scores from 
10th- and llth-grade students was small, as was the number of scores. 

Grade-by-Grade Score Improvement on a “Mock API” from 1999 to 2002 



Average increase in mock API for all students & schools 




Results for disadvantaged students also show noncharters doing better 

Rogosa also compared the performance of socioeconomically disadvantaged 
students based on the concentration of poverty in their schools. He found that 
those attending noncharters tended to show greater gains. Whether the disad- 
vantaged students were in the minority or majority at their school, they tended 
to do better in noncharter schools-quite substantially in some grades. Taking 



all schools into consideration, disadvantaged noncharter students showed 
more improvement than charter students in eight of 10 grades tested (3-39 
more points). 

Average increase in mock API for socioeconomically 




Among schools in which at least half of students were disadvantaged, nonchar- 
ter schools showed more improvement with their disadvantaged students in 
seven of 10 grades (3-51 more points). 



Average increase in mock API for socioeconomically 
disadvantaged students in predominantly poor schools 




Grade 



Charter schools look better in simulation of schoolwide APIs 

Rogosa also simulated schoolwide APIs by aggregating grade-level scores 
into groupings of grades 2-6, 2-8, 9-11, and 2-11. With this level of analy- 
sis, the performance gap between charters and noncharters narrowed-and 
in some cases reversed. For example, when test scores for all charter school 
students in grades 2-6 are combined to produce API scores for 1999 
through 2002, the students gained 76 API points over the three years versus 
74 points for noncharter students. However, students in charter high schools 
showed 15 points of growth while noncharter high school students gained 
20 points. 

A copy of this study is available at: 

www-stat.stanford.edu/-rag/api/charter9902.pdf 
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• | To Learn More 

Studies of charter school performance 

Various national and California studies are described on pages 21 to 23 of this report. Website 
addresses indicate where copies of these studies are available. 

Data about individual charter schools in California 

The Ed-Data Partnership website-at www.ed-data.kl2.ca.us-provides a wealth of data about 
every charter school in California, including student background, staffing information, and 
summary AYP and API reports. Data as far back as 1992-93 is available on the site. 

In addition, a new “Compare Schools” feature on Ed-Data enables you to develop customized 
reports comparing schools you select. You can also use the “Highest/Lowest" feature to create 
lists of California charter schools you would like to see. For example, you can create a list of the 
20 charter high schools with the highest enrollments, or all of the elementary charters that have 
100% fully credentialed teachers. 

Information on charter school laws and policies 

• The charter school section of EdSource Online, which includes an overview, relevant data, and a 
list of EdSource publications related to charter schools is at: www.edsource.org/edu_chart.cfm 

• The California Charter Schools Association can be found at: www.charterassociation.org 

• The Charter Schools Development Center’s website is at: www.cacharterschools.org 
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